Multi-threaded server architecture supporting dynamic reconfiguration

ABSTRACT

A server is dynamically reconfigured by storing a plurality of server configurations in a configuration store. Requests, received by the server, are routed to one of a plurality of workers for processing the requests. Each request is associated with a current configuration of the plurality of configurations that a worker uses to process the request. The number of workers using each configuration of the plurality of configurations is counted. Responsive to the counting, it is determined that a prior configuration of the plurality of configurations is not being used by the workers. The prior configuration is deleted from the configuration store responsive to the determination that the prior configuration is not being used.

BACKGROUND 1. Field of the Invention

This invention pertains in general to web server architectures and inparticular to a multi-threaded server architecture supporting dynamicreconfiguration.

2. Description of the Related Art

A traditional web server architecture has a configuration that describescharacteristics of the web server. The web server configuration candynamically change when, for example, hardware or software associatedwith the web server changes. In addition, the web server includes one ormore web applications for processing requests for resources according tothe web server's current configuration. A web application can processmultiple requests simultaneously and the amount of time required for theweb application to process a given request can vary depending upon thetype of requests and other characteristics.

When the web server configuration changes, newly-received resourcerequests cannot be assigned to existing instances of the webapplications because the applications are processing requests accordingto the previous configuration. Therefore, the web server mustinstantiate new web applications for processing requests using the newcurrent configuration of the server. In addition, the web server mustmaintain the old web applications while the applications processrequests made under the old configuration.

It is computationally inefficient to maintain two or more sets of webapplications for processing resource requests under differentconfigurations. Each set of web applications occupies memory and otherresources of the web server. Therefore, the performance of the webserver is impacted due to the dynamic reconfigurations.

SUMMARY

The above and other needs are met by methods, computer-readable storagemedia, and systems for providing dynamic reconfiguration of amulti-threaded server architecture.

One aspect provides a computer-implemented method for dynamicallyreconfiguring a server by storing a plurality of server configurationsin a configuration store. Requests, received by the server, are routedto one of a plurality of workers for processing the requests. Eachrequest is associated with a current configuration of the plurality ofconfigurations that a worker uses to process the request. The number ofworkers using each configuration of the plurality of configurations iscounted. Responsive to the counting, it is determined that a priorconfiguration of the plurality of configurations is not being used bythe workers. The prior configuration is deleted from the configurationstore responsive to the determination that the prior configuration isnot being used.

Another aspect provides a non-transitory computer-readable storagemedium storing executable computer program instructions for providingdynamic reconfiguration of a multi-threaded server architecture. Thecomputer program instructions store a plurality of server configurationsin a configuration store. Requests, received by the server, are routedto one of a plurality of workers for processing the requests. Eachrequest is associated with a current configuration of the plurality ofconfigurations that a worker uses to process the request. The number ofworkers using each configuration of the plurality of configurations iscounted. Responsive to the counting, it is determined that a priorconfiguration of the plurality of configurations is not being used bythe workers. The prior configuration is deleted from the configurationstore responsive to the determination that the prior configuration isnot being used.

Still another aspect provides a system for providing dynamicreconfiguration of a multi-threaded server architecture. The systemincludes a computer processor and a non-transitory computer-readablestorage medium storing executable computer program instructions thatwhen executed by the computer processor perform actions includingstoring a plurality of server configurations in a configuration store.Requests, received by the server, are routed to one of a plurality ofworkers for processing the requests. Each request is associated with acurrent configuration of the plurality of configurations that a workeruses to process the request. The number of workers using eachconfiguration of the plurality of configurations is counted. Responsiveto the counting, it is determined that a prior configuration of theplurality of configurations is not being used by the workers. The priorconfiguration is deleted from the configuration store responsive to thedetermination that the prior configuration is not being used.

The features and advantages described in this summary and the followingdetailed description are not all-inclusive. Many additional features andadvantages will be apparent to one of ordinary skill in the art in viewof the drawings, specification, and claims hereof.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure (FIG. 1 is a high-level block diagram of a computing environmentsupporting dynamic server reconfiguration, according to one embodiment.

FIG. 2 is a high-level block diagram of a multi-threaded serverarchitecture supporting dynamic reconfiguration, according to oneembodiment.

FIG. 3 is a high-level block diagram illustrating routing of requests toworker threads supporting multiple different router configurationsaccording to one embodiment.

FIG. 4 is a flowchart illustrating steps performed to support dynamicreconfiguration in a multi-threaded server architecture according to oneembodiment.

FIG. 5 illustrates components of an example machine able to readinstructions to dynamically reconfigure a multi-threaded serverarchitecture from a machine-readable medium and execute them in aprocessor or controller, according to one embodiment.

The figures depict an embodiment of the invention for purposes ofillustration only. One skilled in the art will readily recognize fromthe following description that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles of the invention described herein.

DETAILED DESCRIPTION Computing Environment Supporting Dynamic ServerReconfiguration

FIG. 1 is a high-level block diagram of a computing environment 100having a web server 128 supporting dynamic reconfiguration, according toone embodiment. FIG. 1 illustrates multiple client devices 104 and a webserver 128 connected by a network 116. While only client devices 104 andone web server are shown in FIG. 1, embodiments of the computingenvironment 100 can have many such entities connected to the network.

A client device 104 is an electronic device used by a user to performfunctions such as consuming digital content, executing softwareapplications, browsing web sites hosted by or otherwise interacting withthe web server 128 on the network 116, and downloading files. Forexample, the client device 104 may be a smartphone or a tablet,notebook, or desktop computer. In addition, the client device 104 may bean Internet-of-Things (IoT)-connected device such as a home appliance,or even another web server. The client device 104 may include a displaydevice on which the user may view digital content stored on the clientdevice 104 or downloaded from the web server 128. In addition, theclient device 104 may include a user interface (UI), such as physicaland/or on-screen buttons, with which the user may interact to performfunctions such as consuming digital content, obtaining digital content,and transmitting digital content.

A client device 104 sends requests 108 to the web server 128 via thenetwork 116. The request 108 seeks to access a resource maintained,controlled, or otherwise accessible by the web server 128. In oneembodiment, the client device 104 sends the request 108 using a networkprotocol, e.g., the Hypertext Transfer Protocol (HTTP) or a securevariant thereof. For example, a web browser on the client device 104 maysend a request 108 to the web server 128 to fetch or post a file (e.g.,a web page or an image). The request 108 includes informationidentifying the requested resource and may also include informationidentifying the content to be posted, the client device 104, the server128, and the session. For example, the request may include InternetProtocol (IP) addresses and port numbers of the client device 104 andserver 128, a path to the requested resource at the server 128, and acookie identifying the web browsing session.

The network 116 enables communications among the client devices 104 andthe web server 128. To this end, the network 116 receives requests 108and corresponding data (e.g., contents of a file to be posted on a webpage) from client devices 104 and forwards the requests 120 to the webserver 128. Likewise, the network 116 receives responses 124 from theweb server 128 and forwards the responses 112 to the client devices 104.

The network 116 can comprise the Internet as well as mobile telephonenetworks. In one embodiment, the network 116 uses standardcommunications technologies and/or protocols. Thus, the network 116 caninclude links using technologies such as Ethernet, 802.11, Long-TermEvolution (LTE), etc. The networking protocols used on the network 116can include multiprotocol label switching (MPLS), the transmissioncontrol protocol/Internet protocol (TCP/IP), the User Datagram Protocol(UDP), HTTP, the simple mail transfer protocol (SMTP), the file transferprotocol (FTP), etc. The data exchanged over the network 116 can berepresented using technologies and/or formats including the hypertextmarkup language (HTML), the extensible markup language (XML), etc. Inaddition, all or some of links can be encrypted using conventionalencryption technologies such as secure sockets layer (SSL), transportlayer security (TLS), virtual private networks (VPNs), Internet Protocolsecurity (IPsec), etc. In another embodiment, the entities can usecustom and/or dedicated data communications technologies instead of, orin addition to, the ones described above.

The web server 128 receives and processes requests 120 from the clientdevices 104 and sends responses 124 back to the requesting clientdevices 104. As mentioned above, the requests 120 received by the webserver 128 are typically request resources. For a given request 120, theweb server 128 obtains the requested resource and sends a response 124providing the resource back to the requesting client device 104. Theresource is typically a file or other data, such as a web page orcomponent thereof.

In one embodiment, requests 120 received by the web server 128 areprocessed by one or more router modules 132. The router module 132analyzes the requests 120 and routes the requests 120 to one or moreworker threads for further processing. Thus, in the web environment therouter module 132 serves as a HTTP router within the web server 128.There may be multiple router modules 132 operating concurrently in orderto support load balancing and other features.

The router module 132 has a configuration that describes characteristicsof the router 132. The configuration is dynamic and can change over timedue to actions of a web server 128 administrator or automated processes.For example, the ports on which services are bound may change as theadministrator starts and stops various services. The configurationactive for a point in time is said to be “current” for that point intime. As the configurations change, current configurations aresuperseded and become “old” or “prior” configurations.

When the router module 132 receives a request 120, it assigns therequest to a worker thread and also provides the thread with a referenceto the current router configuration for the request (i.e., theconfiguration that was current at the time the request was received).The router module 132 may assign multiple requests to a given workerthread, and the requests assigned to the given worker thread may beassociated with different router configurations. The worker threads, inturn, process the assigned requests 120 using the requests' associatedconfigurations.

The web server 128 can thus efficiently support dynamic reconfiguration.A given worker thread can simultaneously handle requests havingdifferent associated router configurations. There is no need to maintainseparate sets of worker threads for different router configurations. Asa result, server computing resources are not unduly impacted byconfiguration changes and the web server 128 can continue to efficientlyprocess requests through dynamic configuration changes.

Multi-Threaded Server Architecture Supporting Dynamic Reconfiguration

FIG. 2 is a high-level block diagram of a multi-threaded serverarchitecture 200 supporting dynamic reconfiguration according to oneembodiment. FIG. 2 illustrates the architecture 200 of the web server128 of FIG. 1. As shown, the web server 128 includes the router module132, a configuration module 208, a worker module 210, a proxy module216, and a configuration store 218. Other embodiments of the web server128 can have different and/or other modules than the ones describedhere, and the functionalities can be distributed among the modules in adifferent manner.

The configuration store 218 stores current and prior configurations usedby the router module 132. A configuration describes how variousadjustable characteristics of the router module 132 are configured. Forexample, the configuration may specify the IP addresses, host names,server ports, and/or certificates used by the router module 132. Theconfiguration may be represented as a binary vector (e.g., a string ofbits) describing settings for various adjustable characteristics of therouter module 132. In addition, the configuration store 218 storesconfiguration metadata in association with each configuration. In oneembodiment, the configuration metadata includes a global counterindicating the number of worker threads using a configuration. Themetadata may also describe a window in time in which an associatedconfiguration was current and/or other information about theconfiguration.

Depending upon the embodiment, the configuration store 218 may berepresented by a table in memory of the web server 128, within arelational or flat database, and/or using other techniques. Eachconfiguration is stored in a location in which it can be accessed byother entities within the web server 128. For example, eachconfiguration may be stored at a location specified by a memory pointer,file handle, or other type of reference.

The configuration module 208 maintains the router configuration. In thisrole, the configuration module 208 maintains the current configurationand detects changes to the configuration. When the configuration changesinto a new configuration, the configuration module 208 identifies thenew configuration as the current configuration. In one embodiment, theconfiguration module 208 detects a new configuration by receiving theconfiguration in an application programming interface (API) call made tothe web server 128 using an administrative interface.

The configuration module 208 also maintains the configuration store 218.In one embodiment, the configuration module 208 writes the currentconfiguration to the configuration store 218 and maintains a referenceto the current configuration. When the configuration changes, theconfiguration module 208 writes the new configuration to theconfiguration store 218 and updates the reference to point to the newconfiguration. Thus, the configuration module 208 always maintains areference to the location of the current configuration stored within theconfiguration store 218. The configuration module 208 also maintainsreferences to prior configurations, and can delete prior configurationsfrom the configuration store 218 in order to free space for storingother configurations. In one embodiment, the configuration module 208deletes a router configuration from the configuration store 218 if theconfiguration is not the current configuration and the global counterhas a value of zero.

The router module 132, mentioned briefly with respect to FIG. 1,supervises processing of requests from the client devices 104. When aninbound request 120 reaches the web server 128, it is passed to therouter module 132. The router module 132 analyzes the request 120 androutes the request 120 to one or more worker threads of the workermodule 210. Depending upon the request 120, the router module 132 mayrender one or more sub-requests responsive to a given inbound request120 and route these sub-requests to multiple different worker threads.The router module 132 may perform load balancing to distribute requests120 across the multiple worker threads to increase throughput and reducelatency. The router module 132 may use routing policies to control theflow and assignment of requests. The analysis and routing is dependentupon the current configuration of the router module 132.

The router module 132 also associates each routed request 120 (and/orsub-request) with the reference to the location in the configurationstore 210 storing the current configuration at the time the request wasprocessed by the router module 132. Thus, each request routed by therouter module 132 to the worker module 210 also includes an associatedreference to the configuration to be used when the worker module 210processes that request.

The router module 132 also receives results (e.g., resources) producedby the worker module 210 in response to the requests. The router module132 creates outbound responses 124 which it sends to client devices 104in fulfillment of inbound requests. As part of this processing, therouter module 132 may merge multiple sub-responses received from theworker module 210 into a single outbound response.

The worker module 210 represents a pool of worker threads that processthe requests 120 received by the router module 132 according to thespecified router configurations. In one embodiment, the worker module210 represents a set of web applications for processing multipledifferent types of requests. The web applications may be structured asmicroservices, e.g., a collection of loosely coupled services. The webapplications are implemented as a pool of worker threads that processthe various types of requests. The worker threads are used to achieveconcurrency of processing of the requests by processing many requestssimultaneously. The number of worker threads in the pool maintained bythe worker module 210 may be tuned to the computing resources availableto the web server 128, e.g., parallel processors, cores, memory, andnetwork sockets.

Each worker thread processes a request 120 using the routerconfiguration associated with the request 120. The worker thread usesthe reference to the configuration associated with the request 120 toobtain the configuration from the configuration store 218, and then usesthe obtained configuration while processing the request 120. Given thata single worker thread may concurrently process multiple requests 120,and different requests 120 may have different associated routerconfigurations, the worker thread may concurrently process differentrequests 120 using different configurations.

In this manner, dynamic reconfiguration of the web server 128 isattained using the pool of worker threads to simultaneously processdifferent requests 120 using different router configurations. Changingconfigurations does not require worker threads to be deleted or newworker threads to be created. This technique thus increases web server128 performance and reduces latency in execution by avoiding frequentcreation and destruction of worker threads. Different requests 120assigned to different configurations in memory and processed by a singleworker thread can share the memory space allocated to the worker thread.Further, different requests 120 assigned to different configurations inmemory and processed by different worker threads can also share thememory space allocated to the worker module 210. The technique thereforeperforms processing of web applications more efficiently and reducesresource requirements by assigning each request 120 to a worker threadthat lives over the course of more than one request.

The proxy module 216 functions as an intermediary between the workerthreads of the worker module 210 and the configuration store 218. Theproxy module 216 serves to enable counters relating to how the routerconfigurations in the configuration store 218 are being used. Thesecounters include local counters associated with particular workerthreads and the global counters that count the number of worker threadsusing each particular router configuration.

The proxy module 216 separately interfaces with each worker thread. Fora given worker thread, the proxy module 216, maintains a set of localcounters. Each local counter in the set is associated with a routerconfiguration being used by at least one request 120 currently beingprocessed by the associated worker thread. A local counter counts thenumber of requests 120 the worker thread is processing that use that theassociated router configuration. For example, assume a worker thread isconcurrently processing ten requests 120, where six requests 120 use afirst (current) configuration and four requests 120 use a second (prior)configuration. For this thread, the proxy module 216 maintains a localcounter for the first configuration having a value of six and a localcounter for the second configuration having a value of four.

In one embodiment, the worker threads directly update the associatedlocal counters in the proxy module 216. The worker threads increment thelocal counters as the threads receive new requests 120 using givenrouter configurations, and decrement the local counters as the threadscomplete processing requests 120 using given router configurations.Since the proxy module 216 maintains a set of local counters for eachworker thread, no race conditions arise because each of the values ineach set of local counters is changed by only the associated thread.

In addition, the worker threads update the global counters in theconfiguration store 218. In embodiments, a worker thread changes thevalue of a global counter for a router configuration when the thread'slocal counter for that configuration transitions from zero to one orfrom one to zero. These transitions indicate that the worker thread hasstarted or stopped using the router configuration, and therefore theglobal counter, which tracks the number of worker threads using theconfiguration, should be changed. A global counter, in contrast to alocal counter, is subject to race conditions and could become invalid iftwo local threads simultaneously attempted to change it. Therefore, aworker thread obtains a lock for a global counter before changing thecounter's value. The lock prevents two worker threads from changing theglobal counter at the same time. Alternatively, the changing of a globalcounter could be implemented using atomic operations without locking.

The worker threads interact with the proxy module 216 using the routerconfiguration references supplied to the worker threads by the routermodule 132. The reference supplied by router module 132 actuallyreferences a location managed by the proxy module 216 instead of alocation in the configuration store 218. Thus, when a worker threadfollows the reference to obtain the router configuration, it actuallyreferences a location in or managed by the proxy module 216. At thislocation, the proxy module 216 stores a local counter for the workerthread/configuration and a reference to the actual location of therouter configuration and associated global counter in the configurationstore 218. Thus, the worker thread uses the configuration referencesupplied by the router module with the request to access the localcounter for the configuration in the proxy module 216, and uses thereference stored in the proxy module 216 to obtain the configurationfrom the configuration store 218.

In one embodiment, the configuration module 208 maintains the referencesto the configuration store 218 held by the proxy module 216. Asmentioned above, in embodiments, the configuration module 208 may alsomonitor the global counters in the configuration store 218 and deletenon-current configurations having global counter values of zero. Thisway, the configuration store 218 does not store obsolete configurations,and the web server 128 resources are used efficiently.

Routing Requests to Worker Threads

FIG. 3 is a high-level block diagram illustrating routing of requests toworker threads supporting multiple different router configurationsaccording to one embodiment. FIG. 3 uses like reference numerals toidentify like elements. A letter after a reference numeral, such as “310a,” indicates that the text refers specifically to the element havingthat particular reference numeral. A reference numeral in the textwithout a following letter, such as “310,” refers to any or all of theelements in the figures bearing that reference numeral, e.g., “310” inthe text refers to reference numerals “310 a” and/or “310 b” in thefigures.

FIG. 3 illustrates three worker threads 310 a, 310 b, and 310 c, theproxy module 216, and the configuration store 218. Local counters (e.g.,Count1 b) are shown using dashed lines within the associated workerthreads 310 for clarity. In one embodiment the local counters areactually maintained within the proxy module 216.

Assume for purposes of FIG. 3 that the router module 132 routes a firstrequest Request 1 to worker thread 310 b for processing. Request 1 isassociated with router configuration Config1 because this is the currentconfiguration when the request is routed. The local counter Count1 bcorresponding to the worker thread 310 b and the current configurationConfig1 is incremented to one because only one request (Request 1) isassigned to worker thread 310 b and to configuration Config1. The workerthread 310 b uses the reference supplied by the router module 132 toreference the proxy module 216, which in turns references configurationConfig1 on the configuration store 218 for processing request Request1.The worker thread 310 b increments the global counter Count1corresponding to Config1 by one to reflect that a new worker thread 310b, which was not processing requests assigned to Config1 earlier, is nowprocessing a request Request1 assigned to Config1.

Assume that a second request Request2 is also assigned to the currentconfiguration Config1 and to worker thread 310 b for processing. Thelocal counter Count1 b corresponding to the worker thread 310 b and thecurrent configuration Config1 is incremented to two because two requests(Request 1 and Request2) are assigned to worker thread 310 b and toconfiguration Config1. The worker thread 310 b references the proxymodule 216, which in turn references configuration Config1 on theconfiguration store 218 for processing request Request2. The globalcounter Count1 corresponding to Config1 is not incremented becauseworker thread 310 b is already processing a request (Request1) assignedto Config1. The count Count1 of worker threads processing requestsassigned to configuration Config1 is therefore one.

Now, the current configuration of the web server 128 changes fromConfig1 to Config2, and configuration Config1 becomes a previousconfiguration stored by the configuration store 218. When the thirdrequest Request3 is received, it is assigned the current configurationConfig2 and to worker thread 310 a for processing. The local counterCount2 a corresponding to the worker thread 310 a and the currentconfiguration Config2 is incremented to one because only one request(Request 3) is assigned to worker thread 310 a and to configurationConfig2. The worker thread 310 a references the proxy module 216, whichin turns references configuration Config2 on the configuration store 218for processing request Request3. The global counter Count2 correspondingto Config2 is incremented to one to reflect that a new worker thread 310a, which was not processing requests assigned to Config2 earlier is nowprocessing a request Request3 assigned to Config2.

When request Request4 is received, it is assigned the currentconfiguration Config2 and to worker thread 310 c for processing. Thelocal counter Count2 c corresponding to the worker thread 310 c and thecurrent configuration Config2 is incremented to one because one request(Request4) is assigned to worker thread 310 c and to configurationConfig2. The worker thread 310 c references the proxy module 216, whichin turns references configuration Config2 on the configuration store 218for processing request Request4. The global counter Count2 correspondingto Config2 is incremented to two to reflect that a new worker thread 310c, which was not processing requests assigned to Config2 earlier is nowprocessing a request Request4 assigned to Config2.

Worker thread 310 b completes processing Request1 using configurationConfig1. The local counter Count1 b corresponding to the worker thread310 b and to configuration Config1 is decremented to one becauseRequest1 has completed and one other request (Request2) is assigned toworker thread 310 b is using configuration Config1. The global counterCount1 corresponding to Config1 is not decremented because a workerthread (310 b) is still processing requests (Request2) assigned toconfiguration Config1.

The current configuration of the web server 128 changes from Config2 toConfig3, and configuration Config2 becomes a previous configurationstored by the configuration store 218. When the request Request5 isreceived, it is assigned to the current configuration Config3 and toworker thread 310 a for processing. The local counter Count3 acorresponding to the worker thread 310 a and the current configurationConfig3 is incremented to one because only one request (Request 5)assigned to worker thread 310 a is using configuration Config3. Theworker thread 310 a references the proxy module 216, which in turnsreferences configuration Config3 on the configuration store 218 forprocessing request Request5. The global counter Count3 corresponding toConfig3 is incremented to one to reflect that a new worker thread 310 a,which was not processing requests assigned to Config3 earlier is nowprocessing a request Request5 assigned to Config3.

The benefits and advantages of the approach are that incoming requestsmay be assigned to new current configurations while previous requests,assigned to previous configurations, are simultaneously processingwithout the need for deleting worker threads, creating new workerthreads, or delaying new requests until previous requests complete andthe web server 128 is reconfigured. Using local counters eliminates lockcontention between worker threads while accessing a configuration. Eachworker thread is able to process multiple requests assigned to multipleconfigurations simultaneously, thereby improving the distribution ofworkloads across multiple computing resources and reducing resource use,increasing throughput, reducing response time, and avoiding the overloadof any single resource.

Process for Dynamic Reconfiguration of Multi-Threaded ServerArchitecture

FIG. 4 is a flowchart illustrating steps performed to support dynamicreconfiguration in a multi-threaded server architecture according to oneembodiment. FIG. 4 shows certain steps performed by one instance of theweb server 128 described in the preceding figures. Other embodimentsperform the illustrated steps in different orders, and/or performdifferent or additional steps.

Assume for purposes of FIG. 4 that the web server 128 is active and arouter module 132 within the web server is routing 410 requests 120 fromclient devices 104. The router module 132 has a current configurationstored in the configuration store 218 with which it associates therouted requests 120. The routed requests 120 are processed by workerthreads which use the associated configurations. Concurrently with therequest routing 410, the web server 128 monitors 412 for changes to theconfiguration of the router module 132. For example, the web server 128monitors for any router configuration changes received via an API of theadministrative interface to web server 128. Through this monitoring, theweb server detects 412 a change to the configuration of the routermodule 132. The web server 128 stores 412 the changed configuration inthe configuration store 218 as the new current configuration for therouter module 132. The web server 128 also continues to store the priorconfiguration in the configuration store 218. The router module 132continues to route 410 requests 120 received after the configurationchange, and associates these requests 120 with the new currentconfiguration.

The web server 128 updates 414 counters counting usage of the routerconfigurations by the worker threads. Collectively, the worker threadsare processing requests using the current configuration and also may beprocessing requests using prior configurations. The counters includeglobal counters that count the number of worker threads using particularconfigurations (e.g., the global counter for the current configurationcounts the number of different worker threads that are processing atleast one request using the current configuration). In addition, thecounters include local counters counting the number of requests eachworker thread is processing using particular configurations. In oneembodiment, each worker thread is associated with its own set of localcounters, where the set contains one counter for each configurationbeing used by the worker thread, and each counter counts the number ofrequests being processed by the worker thread using a givenconfiguration. The web server 128 updates the global and local countersby appropriately incrementing and decrementing the counters as newrequests are routed to the worker threads and the worker threadscomplete processing of older requests.

The web server 128 deletes 416 prior configurations from theconfiguration store 218 once the configurations are no longer being usedby the worker threads. The web server 128 monitors the global countersfor the prior configurations to determine the number of worker threadsthat are using the configurations. When a global counter for a priorrouter configuration decrements to zero, this means no worker threadsare using the configuration. Therefore, the web server 128 deletes theconfiguration from the configuration store 218. Deleting theconfigurations in this way prevents the configuration store 218 fromgrowing unduly large as the router configurations change over time.

Example Machine Providing Dynamic Reconfiguration of Multi-ThreadedServer Architecture

FIG. 5 illustrates components of an example machine 500 able to readinstructions to dynamically reconfigure a multi-threaded serverarchitecture from a machine-readable medium and execute them in aprocessor or controller, according to one embodiment. Those of skill inthe art will recognize that other embodiments of the machine 500 canhave different and/or other modules than the ones described here, andthat the functionalities can be distributed among the modules in adifferent manner.

Specifically, FIG. 5 shows a diagrammatic representation of a machine inthe example form of a computer system 500. The computer system 500 canbe used to execute instructions 524 (e.g., program code modules) thatcause the machine to perform any one or more of the methodologies (orprocesses) described herein. In alternative embodiments, the machineoperates as a standalone device or a connected (e.g., networked) devicethat connects to other machines. In a networked deployment, the machinemay operate in the capacity of a server machine or a client machine in aserver-client network environment, or as a peer machine in apeer-to-peer (or distributed) network environment.

The machine may be a server computer, a cloud server residing on ashared “virtualized” environment managed by a cloud hosting provider, apersonal computer (PC), a tablet PC, a set-top box (STB), a smartphone,an internet of things (IoT) appliance, a network router, switch orbridge, or any machine capable of executing instructions 524 (sequentialor otherwise) that specify actions to be taken by that machine. Further,while only a single machine is illustrated, the term “machine” shallalso be taken to include any collection of machines that individually orjointly execute instructions 524 to perform any one or more of themethodologies discussed herein.

The example computer system 500 includes one or more processing units(generally processor 502). The processor 502 is, for example, a centralprocessing unit (CPU), a graphics processing unit (GPU), a digitalsignal processor (DSP), a controller, a state machine, one or moreapplication specific integrated circuits (ASICs), one or moreradio-frequency integrated circuits (RFICs), or any combination ofthese. The computer system 500 also includes a main memory 504. Thecomputer system may include a storage unit 516. The processor 502,memory 504 and the storage unit 516 communicate via a bus 508.

In addition, the computer system 500 can include a static memory 506, adisplay driver 510 (e.g., to drive a plasma display panel (PDP), aliquid crystal display (LCD), or a projector). The computer system 500may also include alphanumeric input device 512 (e.g., a keyboard), acursor control device 514 (e.g., a mouse, a trackball, a joystick, amotion sensor, or other pointing instrument), a signal generation device518 (e.g., a speaker), and a network interface device 520, which alsoare configured to communicate via the bus 508.

The storage unit 516 includes a machine-readable medium 522 on which isstored instructions 524 (e.g., program code modules) embodying any oneor more of the methodologies or functions described herein. Theinstructions 524 may also reside, completely or at least partially,within the main memory 504 or within the processor 502 (e.g., within aprocessor's cache memory) during execution thereof by the computersystem 500, the main memory 504 and the processor 502 also constitutingmachine-readable media. The instructions 524 may be transmitted orreceived over a network 526 via the network interface device 520.

While machine-readable medium 522 is shown in an example embodiment tobe a single medium, the term “machine-readable medium” should be takento include a single medium or multiple media (e.g., a centralized ordistributed database, or associated caches and servers) able to storethe instructions 524. The term “machine-readable medium” shall also betaken to include any non-transitory medium that is capable of storinginstructions 524 for execution by the machine and that cause the machineto perform any one or more of the methodologies disclosed herein. Theterm “machine-readable medium” includes, but not be limited to, datarepositories in the form of solid-state memories, optical media, andmagnetic media.

The above description is included to illustrate the operation of thepreferred embodiments and is not meant to limit the scope of theinvention. The scope of the invention is to be limited only by thefollowing claims. From the above discussion, many variations will beapparent to one skilled in the relevant art that would yet beencompassed by the spirit and scope of the invention.

What is claimed is:
 1. A computer-implemented method of dynamicallyreconfiguring a server, comprising: storing a plurality of serverconfigurations in a configuration store; routing requests received bythe server to one of a plurality of workers for processing the requests,each request associated with a current configuration of the plurality ofconfigurations that a worker uses to process the request; counting anumber of workers using each configuration of the plurality ofconfigurations; determining, responsive to the counting, that a priorconfiguration of the plurality of configurations is not being used bythe workers; and deleting the prior configuration from the configurationstore responsive to the determination that the prior configuration isnot being used.
 2. The computer-implemented method of claim 1, whereinthe plurality of configurations comprise a current configuration and oneor more prior configurations, and wherein the requests routed to one ofthe plurality of workers are associated with the current configuration.3. The computer-implemented method of claim 1, further comprising:detecting a change to a server configuration used by the server, thechange producing a new configuration of the server; storing the newconfiguration in the configuration store; and representing the newconfiguration as the current configuration.
 4. The computer-implementedmethod of claim 1, wherein the routing of the requests comprises:determining a reference to a location of the current configuration inthe configuration store; and providing the reference to the location ofthe current configuration to a worker to which a request is routed. 5.The computer-implemented method of claim 4, wherein the determining ofthe reference to the location comprises: determining a reference to alocation at a proxy within the server, the proxy storing the location ofthe current configuration in the configuration store.
 6. Thecomputer-implemented method of claim 1, wherein the counting of thenumber of the workers comprises: maintaining a plurality of globalcounters, each global counter associated with a configuration of theplurality of configurations and counting a number of workers currentlyprocessing requests using the configuration; wherein the plurality ofglobal counters are updated by the number of workers.
 7. Thecomputer-implemented method of claim 1, wherein the counting of thenumber of workers comprises: maintaining a plurality of sets of localcounters, each set of local counters associated with a worker of theplurality of workers, and each local counter in a set associated with aconfiguration of the plurality of configurations, wherein each localcounter in the set counts a number of requests being processed by theworker using the associated configuration; and updating, by the worker,the set of local counters associated with the worker.
 8. Anon-transitory computer-readable storage medium storing executablecomputer program instructions for dynamically reconfiguring a server,the computer program instructions comprising instructions for: storing aplurality of server configurations in a configuration store; routingrequests received by the server to one of a plurality of workers forprocessing the requests, each request associated with a currentconfiguration of the plurality of configurations that a worker uses toprocess the request; counting a number of workers using eachconfiguration of the plurality of configurations; determining,responsive to the counting, that a prior configuration of the pluralityof configurations is not being used by the workers; and deleting theprior configuration from the configuration store responsive to thedetermination that the prior configuration is not being used.
 9. Thenon-transitory computer-readable storage medium of claim 8, wherein theplurality of configurations comprise a current configuration and one ormore prior configurations, and wherein the requests routed to one of theplurality of workers are associated with the current configuration. 10.The non-transitory computer-readable storage medium of claim 8, furthercomprising instructions for: detecting a change to a serverconfiguration used by the server, the change producing a newconfiguration of the server; storing the new configuration in theconfiguration store; and representing the new configuration as thecurrent configuration.
 11. The non-transitory computer-readable storagemedium of claim 8, wherein the routing of the requests comprises:determining a reference to a location of the current configuration inthe configuration store; and providing the reference to the location ofthe current configuration to a worker to which a request is routed. 12.The non-transitory computer-readable storage medium of claim 11, whereinthe determining of the reference to the location comprises: determininga reference to a location at a proxy within the server, the proxystoring the location of the current configuration in the configurationstore.
 13. The non-transitory computer-readable storage medium of claim8, wherein the counting of the number of the workers comprises:maintaining a plurality of global counters, each global counterassociated with a configuration of the plurality of configurations andcounting a number of workers currently processing requests using theconfiguration; wherein the plurality of global counters are updated bythe number of workers.
 14. The non-transitory computer-readable storagemedium of claim 8, wherein the counting of the number of workerscomprises: maintaining a plurality of sets of local counters, each setof local counters associated with a worker of the plurality of workers,and each local counter in a set associated with a configuration of theplurality of configurations, wherein each local counter in the setcounts a number of requests being processed by the worker using theassociated configuration; and updating, by the worker, the set of localcounters associated with the worker.
 15. A computer system comprising: acomputer processor; and a non-transitory computer-readable storagemedium storing executable computer program instructions that whenexecuted by the computer processor perform actions comprising: storing aplurality of server configurations in a configuration store; routingrequests received by the server to one of a plurality of workers forprocessing the requests, each request associated with a currentconfiguration of the plurality of configurations that a worker uses toprocess the request; counting a number of workers using eachconfiguration of the plurality of configurations; determining,responsive to the counting, that a prior configuration of the pluralityof configurations is not being used by the workers; and deleting theprior configuration from the configuration store responsive to thedetermination that the prior configuration is not being used.
 16. Thecomputer system of claim 15, wherein the plurality of configurationscomprise a current configuration and one or more prior configurations,and wherein the requests routed to one of the plurality of workers areassociated with the current configuration.
 17. The computer system ofclaim 15, the actions further comprising: detecting a change to a serverconfiguration used by the server, the change producing a newconfiguration of the server; storing the new configuration in theconfiguration store; and representing the new configuration as thecurrent configuration.
 18. The computer system of claim 15, wherein therouting of the requests comprises: determining a reference to a locationof the current configuration in the configuration store; and providingthe reference to the location of the current configuration to a workerto which a request is routed.
 19. The computer system of claim 18,wherein the determining of the reference to the location comprises:determining a reference to a location at a proxy within the server, theproxy storing the location of the current configuration in theconfiguration store.
 20. The computer system of claim 15, wherein thecounting of the number of the workers comprises: maintaining a pluralityof global counters, each global counter associated with a configurationof the plurality of configurations and counting a number of workerscurrently processing requests using the configuration; wherein theplurality of global counters are updated by the number of workers.